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Traditionally, it is assumed that source coding is perfect and therefore, the 
redundancy of the source encoded bit-stream is zero. However, in reality, this 
is not the case as the existing source encoders are imperfect and yield 
residual redundancy at the output. The residual redundancy can be exploited 
by using Joint Source Channel Coding (JSCC) with Markov chain as the 
source. In several studies, the statistical knowledge of the sources has been 
assumed to be perfectly available at the receiver. Although the result was 
better in terms of the BER performance, practically, the source correlation 
knowledge were not always available at the receiver and thus, this could 
affect the reliability of the outcome. The source correlation on all rows and 
columns of the 2D sources were well exploited by using a modified Bahl- 
Cocke-Jelinek-Raviv (BCJR) algorithm in the decoder. A parameter 
estimation technique was used jointly with the decoder to estimate the source 
correlation knowledge. Hence, this research aims to investigate the parameter 


estimation for 2D JSCC system which reflects a practical scenario where the 
source correlation knowledge are not always available. We compare the 
performance of the proposed joint decoding and estimation technique with 
the ideal 2D JSCC system with perfect knowledge of the source correlation 
knowledge. Simulation results reveal that our proposed coding scheme 
performs very close to the ideal 2D JSCC system. 
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1. INTRODUCTION 

Many improvements were implemented to channel coding including turbo coding which was 
introduced in 1993 [1]. The advantages of the turbo codes include high reliability and near to Shannon’s 
capacity performance and hence, make turbo codes to be well adopted in various applications [2]-[4]. 
Shannon’s separation theorem states that an optimal performance in separate source and channel coding 
system can be achieved if channel coding and source coding are optimal with zero delay [5]. However, it is 
impossible to get the finest performance with zero delay as the restriction on latency definitely occurs in 
practical applications. Joint Source and Channel Coding (JSCC) has been introduced by making use of the 
residual redundancies left in source encoders to improve the error correction capability of a channel 
code [6]-[8]. 

Recent research of [9], [10] have introduced 2-Dimensional (2D) source correlation for a JSCC 
system was exploited and utilized in the system design to improve the Bit Error Rate (BER) performance. 
However, these studies assumed that the source correlation knowledge is perfectly known at the receiver 
[9]-[11]. This assumption improves the performance but it affects the reliability of the information of source 
correlation to be exploited at the receiver. Practically, source correlation knowledge is unknown and needs to 
be estimated. Thus, many researchers have taken up this issue and developed various parameter estimation 
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techniques to estimate the source correlation at the receiver to ensure the performance is improved and 
applicable in real practices [12], [13]. 

Several methods of parameter estimation have been presented in various studies based on a Hidden 
Markov Model (HMM) such as the Baum-Welch Algorithm (BWA) [14]-[20]. This enables the decoder to 
utilize the estimated statistics of the source correlation knowledge in channel decoding to improve the system 
performance. In this paper, we propose 2D JSCC system with BWA-based estimation technique to estimate 
the source correlation knowledge at the channel decoder on the non-HMM model to get the performance as 
close as possible to the ideal system. The proposed design is then compared with the existing 2D JSCC 
system with the perfect assumption on source correlation knowledge or known as the ideal 2D JSCC system 
[9] and another technique of parameter estimation exploiting the extrinsic information gleaned from the 
decoders adopted from [11]. 

This paper has been organised in the following way. The related works of parameter estimation 
techniques on channel decoding and the mechanism of BWA are discussed in Section 2. The proposed model 
on the coding scheme is presented in Section 3. Section 4 deliberates on the difference between BER 
performance from the proposed coding scheme and existing work. Finally, Section 5 draws upon the entire 
thesis, gives a conclusion and identifies areas for further research. 


2. RELATED WORKS ON PARAMETER ESTIMATION TECHNIQUES 
2.1. Parameter estimation on turbo decoder 

Several attempts have been made previously to describe the parameter estimation of Markov model 
at the Turbo decoder in [15], [16]. These approaches have known to be complex due to the essence of 
building the supertrellises combining both constituent encoders and the HMM models. The modification on 
the decoder [17] has been done to improve the performance while the simplified method for modifying a 
decoder has been described in [18] to reduce its complexity while maintaining the same performance as in 
[16]. The parameter estimation has been refined in [19] to enhance the connection between BWA as the 
parameter estimation technique and Turbo decoder. BWA is implemented in the first iteration in order to 
have an initial estimation of Hidden Markov source parameters. The correlation pattern is considered and 
characterized by HMM [19]. The expected number of transitions from state to state that generate correlation 
pattern v are represented by the product of the branch probability and the number of trellis transitions. Hence, 
making it easy to estimate the expected state transition. Anwar, in [20], suggested that the stability and 
efficient exploitation of BWA can be improved by using a forward-backward algorithm or the uni-directional 
algorithm. 


2.2. The simplified Baum-Welsh algorithm 

The potential algorithm associated with the learning problem of an HMM is the BWA which has 
been developed in 1970 [21] and also known as Forward-Backward algorithm which is considered as the 
unique case of the Expectation-Maximization or EM algorithm [22]. In the learning problem, the parameters 
of an HMM denoted as are developed and adjusted depending on the applications given an observation 
sequence O and the set of possible states Q. Meanwhile, for the non-HMM considered in this work, we only 
used one parameter to be estimated to reduce the complexity of the system while maintaining its peak 
performance. In this method, only transition probabilities are used to estimate the source statistics in the 
decoder. The equation of transition probability is adopted from the previous work done by Garcia-Frias and 
Villasenor in [19]. The BWA is generally performed by calculated the initial estimation for the probabilities, 
and iteratively uses the information from the estimated value to perform better estimation upon the 
probabilities on the next iteration. The forward variables can be given as: 


æ (i) = P(0,,03,...,0,,4, =| A) , (1) 


where is forward variable, and is the partial observation sequence while the backward variable can be stated 
as: 


Bi) = P(Oj435O,499-+ +9; |a, =1,A) > (2) 


VAO) 0420 o 


where is the probability of the partial sequence “‘t!’“*+2”""*?'“* at the current state /- 
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The forward and backward variables in Equation (1) and Equation (2), respectively can be 
implemented to initialize the transition probability. The transition probability can be estimated as 


â = expected number of transitions from state i to state j 
o 


expected number of transitions from state i (3) 


In order to calculate the numerator of, the joint probability of being in state at time and state at time denoted 
as is defined as: 


EG, j) = Pq, =1.4.4 = J|0,2) >, (4) 
where the observation sequence and are given. The equation also can be written as: 


. ~ _ PG, =i,4 = j|0,⁄) 
7) = ee 5 


By using the forward and backward variables, the final equation of can be modeled as 


= a, (a,b, (Ps (7) 
a, (qF) 


é,(i, j) (6) 


where is the final state. 


Thus, the numerator of the transition probability can be calculated as a summation of for all t while the 
denominator of which is the summation of all the expected number of transitions in state can be denoted The 
final formula for can be stated as 


t=1 k=l (7) 


In summary, the estimation of the transition probability can be re-estimated using Equation (7) again to 
exploit the previous estimated value. The re-estimation process is considered as the basis of the iterative 
forward-backward algorithm. 


3. SYSTEM DESIGN 

Figure | illustrates the top-level design of the proposed communication system. The source natural 
redundancy was considered in this study for simplicity and generalization purposes as there is no source 
encoder employed in the proposed system. The source is a 2D binary source having a 2D source correlation. 
The 2D source is converted into a 1D sequence represented by a vector u before being encoded by a channel 
encoder to output the encoded sequence c. Then, the encoded sequence c is modulated by a BPSK modulator 
to generate modulated output sequence g before being transmitted over an AWGN channel. During the 
modulation process, the binary encoded sequence is transformed into a signal form that can be physically 
transmitted, i.e., bit 0 maps to a signal amplitude -1 and bit 1 maps to a signal amplitude 1. 

In a real environment, distortion and noise always be the factors that disturb the received signal at 
the receiver. Most commonly used and simple model to denote the channel behaviors of a communication 
system is the Additive White Gaussian Noise (AWGN), which is due to the thermal noise produced by the 
electronic hardwares and devices. In this model, the independent Gaussian noise signals with zero mean n 
and variance are added to the transmitted signal g. The energy of the transmitted signal is normalized to 1 and 
the examined energy per bit to noise power spectral density ratio, is used to measure the channel quality. The 
signal will be distorted and sent to the receiver as z = g + n. At the receiver, the received signal sequence z is 
demodulated by a BPSK demodulator to generate an output sequence r. The a posteriori information of the 
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source correlation knowledge is estimated during channel decoding using the proposed parameter estimation 
technique. The new estimated value of source correlation knowledge is exploited at the decoder before 
outputting the decoded binary sequence. The detail of the proposed transmitter and receiver model will be 
presented in the following sections. 


Parameter 


Estimation 


i ! i ' 
u| Channel |€ | Besk E Di]  Bpsk ; i 
i G S Channel i 
' Encoder Modulator | | Demodulator Decoder with j 
' I 1 
i i j 


Transmitter Receiver 


Figure 1. General view of the proposed JSCC design 


3.1. Transmitter design 

The proposed encoder employs two channel codes that are serially concatenated to each otherwhich 
are Turbo Single Parity Check Code (TSPCC) [23], [24] and a rate-1 Recursive Systematic Convolutional 
Code (RSCCs) [25], [26]. The TSPCC acts as an outer code and comprises a concatenation of multiple 
SPCCs arranged in parallel with interleavers separating the SPCCs. A rate-1 RSCC is employed as the inner 
code, to improve the performance of the proposed system and to remove the high error-floor problem 
encountered by TSPCC [23], [24]. 

Figure 2 illustrates the block diagram of the transmitter model for the proposed JSCC system 
exploiting 2D source correlation. The 2D sources of is generated by S. The research target is to come out 
with an estimation technique for the 2D correlation parameters that can work well with the decoder of [9], 
thus the same design is used for the transmitter model. Two component encoders for TSPCC, and are 
considered. The interleaver, is a block interleaver that is used to arrange the source sequence to a sequence 
corresponding to the different direction of the source correlation denoted by. A TSPCC uses SPCCs as its 
component codes and it is known that SPCCs are simple codes that only generate a single parity check bit for 
any length of information bits. The advantage of this code is the flexibility in working with any length of 
information bits and it can operate at a very high coding rate. The parity bit sequences, and and source 
sequences are multiplexed after the source sequences have been encoded by the component codes. This 
joined sequence w is then rearranged by a random interleaver before passed through another encoding 
process by. The final encoded sequence c is modulated using a BPSK modulator before being transmitted 
over an AWGN channel. 


3.2. Receiver design 

The detailed receiver model of the proposed JSCC system is shown in Figure 3. The signal from the 
channel is received and demodulated to generate an output sequence, r. By using Bahl-Cocke-Jelinek-Raviv 
(BCJR) algorithm in, the sequence r is decoded via Maximum A Posteriori (MAP) decoding process [27]. 
Soft information in Log-Likelihood Ratio (LLR) form is used while performing the decoding process. The a 
priori LLR of coded bit, at a time index t, can be computed as: 

$ p AN 

La {w 2) =In P(w,=0) (8) 
where is the probability that has the value of 1, while is the probability of holds the value 0. The extrinsic 
LLR, that is generated from is de-interleaved using before fed into and. The two component decoders 
correspond to the decoding of the horizontal and vertical direction source sequence. Modified BCJR 
algorithm is employed to exploit the source correlation throughout the decoding process [9]. Each component 
decoder yields a posteriori and extrinsic LLRs outputs. The a posteriori are then sent to the parameter 
estimation technique block which employs the BWA given in Equation (7) to estimate the transition 
probabilities where p? ‘= p? 2 = Go and Dr T= p? ? = @,,. The extrinsic LLRs from both activated 
component decoders that correspond to the source sequence u are summed up and multiplexed with all the 
extrinsic LLRs corresponding to the parity bit sequences. The combined sequence is then interleaved by Iin 
before feedback to Ch. 
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This process is repeated for a number of iterations. After the final iteration, the a posteriori 
Lipp from Cz? is deinterleaved by Iip . Hard decisioning is made to the a posteriori LLRs output Lipp to 
generate the decoded bits sequence &. 
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Figure 3. The proposed design at the receiver for the 2D JSCC system with parameter estimation 
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4. RESULT AND ANALYSIS 

In this work, the main goal is to develop an accurate parameter estimation technique to estimate the 
unknown source statistics at the receiver for the 2D JSCC system performing as close as possible to the ideal 
2D JSCC system as proposed in [9]. The BER performance is evaluated, by finding the number of errors of 
the decoded binary sequence from the original binary sequence from the source and this number of errors is 
divided by the frame length of the source sequence [28]. The Markov source was initialized with 0.5 to both 
dimensions. To illustrate the result, a simulation of BER against SNR simulation was performed using 
MATLAB software.The configuration used in the simulations to evaluate the performance of the proposed 
2D JSCC system are given in Table 1. Turbo-Single Parity-Check-Code (TSPCC) was used as the outer code. 
The Single-Parity-Check (SPC) code consisting of 8 codeword length and 7 information bits is used for the 
component code of the outer code. The rate-1 Recursive-Systematic-Convolutional (RSC) code which had 
generator polynomial, (G,.,G) = (3,2), (given in octal numeral system) is used in the inner code where G, is 
the feedback polynomial and G is the feed-forward polynomial. 

Figure 4 illustrates the comparison in BER performance of the proposed 2D JSCC system and the 
ideal 2D JSCC system with source correlation in horizontal and vertical direction varying from = until =. The 
performance of 2D JSCC system assisting with the BWA-based estimation technique or also known as 2D 
JSCC system with Parameter Estimation 1 (2D-JSCC-PET1 system) is very close to the ideal 2D JSCC 
system and improves when the source correlation becomes stronger. For example, at BER level of, an 
improvement of 1.18 dB is achieved is achieved with the proposed 2D-JSCC-PET1 system over the 
non-JSCC system for p = 0.7 and the improvement increases to 4.48 dB for p = 0.9. The performance gain 
between the ideal 2D JSCC system and the proposed 2D-JSCC-PET 1 system is about the same for each value 
of p. As can be seen in the Figure 4, the ideal 2D JSCC system is only slightly better than the proposed 2D- 
JSCC-PET1 system. However, the ideal 2D JSCC system relies on the assumption of perfect source 
knowledge at the receiver, which may not be suitable practically. Usually, the parameter estimation technique 
requires complex computations concerning its accuracy that might lead to performance degradation but 
despite the degradation that may occur, the gain for our proposed 2D-JSCC-PET1 system is not too far from 
the ideal system and hence, ensuring the system performing better in terms of the accuracy and the reliability 
of the data transmission at the receiver. 

In another simulation, the performance difference between the proposed 2D-JSCC-PET1 system and 
the similar system without any parameter estimation denoted as the 2D JSCC system reveals that without the 
estimation technique, the performanceof the system could not do any better for unknown source statistics. 
The gain is measured as a difference of Eb/NO at the BER level of between the 2D JSCC systems and the 
non-JSCC system. The performance gains of different systems are tabulated in Table 2. 

Figure 5 compares the BER performance between the proposed 2D-JSCC-PET1 system and 2D- 
JSCC-PET2 system when the correlation parameters are unknown for sources, exhibiting correlation varying 
from = until =. The performance gain between the proposed 2D-JSCC-PET1 system and the similar 2D JSCC 
system using different parameter estimation or the Parameter Estimation 2 (PET2) is summarized in Table 3. 
Figure 5 indicates that the PET1 module is much better than the PET2 module which simply comparing the 
reliable extrinsic LLRs gathered from each decoder [11]. 

It can be seen from Table 3 that there is a significant difference in BER performance between the 
proposed 2D-JSCC-PET1 system and the 2D-JSCC-PET2 system which indicates that the proposed 2D- 
JSCC-PET1 system is performing better than the 2D-JSCC-PET2 system. Based on the performance in 
Figure 5, it can be seen that the gain difference over the non-JSCC system between the proposed 2D-JSCC- 
PET1 system and the 2D-JSCC-PET2 system becomes larger for stronger source correlation, where at p =0.7, 
the gain difference is 0.84 dB and it increases to 3.47 dB at p =0.9. The performance achieved by the 2D 
JSCC systems with the parameter estimation technique show a significant difference when compared to the 
2D JSCC system without employing any estimation method (the 2D JSCC system) and the non-JSCC 
system. 


Table 1. Default Settings Used for the Simulations of the Proposed 2D JSCC System 


Parameter Value 
Outer Code 2D TSPCC 
Component Codes for Outer Code SPC(8,7) Code 
Inner Code Rate-1 RSC (3,2)s Code 
Frame Length, Kyr 608 x 608 bits 
Il, Block Interleaver: 608 x 608 bits 
Tin Random Interleaver : Length 363,888 bits 
Code Rate, Rc 0.78 
Decoding Iterations 25 
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Figure 4. Comparison in BER performance of the proposed 2D-JSCC-PET1 system and the ideal 2D JSCC 
system [9] as the value of p increases 


Table 2. Performance Gain in dB of the Proposed 2D-JSCC-PET1 System and the Ideal 2D JSCC System 
Over the non-JSCC System 


p The ideal 2D 2D-JSCC- Gain for The ideal 2D Gain for Gain 
JSCC system PET1 JSCC system 2D-JSCC- Difference 

(dB) (dB) (dB) PET! (dB) (dB) 

0.7 1.49 1.54 1.23 1.18 0.05 

0.8 0.29 0.34 2.43 2.38 0.05 

0.9 -1.82 -1.71 4.54 4.48 0.06 
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Figure 5. Performance gain in dB of the proposed 2D-JSCC-PET1 system and the 2D-JSCC-PET2 system 


Table 3. Performance Gain in dB of the Proposed 2D-JSCC-PET 1 System and the 2D-JSCC-PET2 System 
over the non-JSCC System 


p 2D-JSCC- 2D-JSCC-PET2 Gain for 2D- Gain for 2D-JSCC-PET2 (dB) Gain 
PETI (dB) JSCC-PET1 Difference 
(dB) (dB) (dB) 
0.7 1.54 2.38 1.18 0.34 0.84 
0.8 0.34 2.06 2.38 0.64 1.74 
0.9 -1.71 1.71 4.48 1.01 3.47 
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5. CONCLUSION AND FUTURE WORK 

This research proposed a 2D JSCC system exploiting estimation technique based on the Baum- 
Welsh algorithm in the decoder on unknown source statistics (2D-JSCC-PET1). The BWA that is used in the 
proposed design has been detailed with its calculations and equations. For sources with identical , the gain of 
the proposed 2D-JSCC-PET1 system and the ideal 2D JSCC system over the non-JSCC system shows very 
small difference which are 4.54 dB and 4.48 dB, respectively. The results also indicate that the difference in 
performance gain between of the proposed 2D-JSCC-PET1 system and the ideal 2D JSCC system is 
approximately the same for strongest source correlation. The difference of proposed 2D-JSCC-PET1 system 
and ideal 2D JSCC system values at p = 0.7 and p =0.9 are slightly the same i.e. 0.05 dB and 0.06 dB, 
respectively. This concludes that it is possible to get the best performance with the parameter estimation 
technique on unknown source correlation knowledge. The proposed 2D-JSCC-PET1 system is then 
compared to another estimation technique based on a previous study [11] known as 2D-JSCC-PET2 system. 
The difference in performance gain between the proposed 2D-JSCC-PET1 system and the 2D-JSCC-PET2 
system increases with the increment of source correlation, i.e., from a difference of 0.84 dB at p = 0.7 toa 
difference of 3.47 dB at p = 0.9. Future studies could develop and enhance parameter estimation technique in 
2D JSCC system with a higher order of Markov Source and higher dimension of source correlation in order 
to obtain a more reliable and better performance. 
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